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ABSTRACT 


As Unmanned Aerial Vehicles (UAVs) become more prevalent on the battlefield, ground forces 
will have to increasingly rely on them for intelligence, surveillance, and reconnaissance (ISR), 
as well as target marking, and overwatch operations. The Situational Awareness for Surveil¬ 
lance and Interdiction Operations (SASIO) simulation analysis tool uses Design of Experiments 
(DOX) to study of aspects of UAV surveillance characteristics in conjunction with ground-based 
interdiction teams. The goal is to reduce the time required to intercept and capture targets of 
interest. Through screening analysis, significant factors can be determined to build a model that 
will provide a ground commander with insights to aid in the tactical employment of his assets. 
We will examine different teaming strategies and coordination measures between searching and 
interdicting assets in order to study the effectiveness of the interdictor possessing an organic, 
tracker UAV. The objective of this research is to quantify the benefit or penalty of an additional 
UAV asset that is organic to a quick reaction force, in the context of the overall surveillance and 
interdiction operation. 
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Executive Summary 


As Unmanned Aerial Vehicles (UAVs) become more prevalent on the battlefield, ground forces 
will have to increasingly rely on them for intelligence, surveillance, and reconnaissance (ISR), 
as well as target marking, and overwatch operations. The Situational Awareness for Surveil¬ 
lance and Interdiction Operations (SASIO) simulation analysis tool uses Design of Experiments 
(DOX) to allow for the study of aspects of UAVs surveillance characteristics in conjunction with 
ground-based interdiction teams to aid in reducing the time required to intercept and capture 
targets of interest. Through screening analysis, significant factors can be determined to build 
a model that will provide a ground commander with insights to aid in the tactical employment 
of his assets. We will examine different teaming strategies and coordination measures between 
searching and interdicting assets in order to study the effectiveness of the interdictor possessing 
an organic, tracker UAV. The objective of this research is to quantify the benefit or penalty of 
an additional UAV asset that is organic to a quick reaction force, in the context of the overall 
surveillance and interdiction operation. 

Utilizing design of experiments and statistical methods, we focused on identifying the signifi¬ 
cant factors that impact the Percentage of Targets Cleared and the Percentage of Time Surveyor 
UAV Performs Search for three different team types. These team types were (1) Surveyor, (2) 
Surveyor/Tracking, and (3) Surveyor with Tracker. We were able to determine, in all cases, that 
the factors with the greatest effect were the Search Area size and Team Type. Holding all other 
factors constant, we determined that as the Search Area increases, the Percentage of Targets 
Cleared decreases and the Percentage of Time Surveyor UAV Performs Search increases. We 
also discovered that possessing a tracking capability is clearly superior to no tracking capability. 
Of particular interest is that there was no significant difference between the Surveyor/Tracking 
and Surveyor with Tracker Team Types. Therefore, we conclude that possessing an augmented 
tracking capability does not increase the Percentage of Targets Cleared. 

We also discovered that the Interdictor Transit Time was a significant factor for our model. The 
time required to reach targets of interest was the limiting factor in clearing those targets. We 
recommend studying various forms of mobility with differing speeds to determine which would 
increase the Percentage of Targets Cleared. Through field experimentation at Camp Roberts 
CA, we identified various real-world constraints that effect the response variables. Specifically, 
weather considerations and QRF proficiency in UAV operations will impact the responses. Nei- 




ther of these considerations were modeled for our simulation and are worthy of future study. 


The SASIO model can be used for any future studies involving teaming. It is recommended for 
future Naval studies in the use of Unmanned Underwater Vehicles (UUVs) and Marine Corps 
studies in the use of Cargo UAS as well as Unmanned Ground Vehicles (UGVs). 
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CHAPTER 1: 
Introduction 


1.1 Background 

The Marine Corps Vision and Strategy 2025 [1] clearly details the Commandant’s vision of the 
future Corps and describes his plan for creating the Marine Corps of 2025. Two important facets 
of this vision state the Marine Corps will deploy forward with relevant and timely capabilities 
and will be lean, agile, and adaptable. As part of this vision, the Deputy Commandant for Avi¬ 
ation has outlined goals to ensure the Aviation Combat Element (ACE) of the Marine Corps is 
aligned with Strategy 2025. Two of these goals are : (1) execute the planned Type/Model/Series 
transition strategies from the legacy platforms to the next generation platforms, and (2) improve 
warfighting integration by developing new concepts of operations to leverage our transforma¬ 
tional systems. 

To support these goals, the Marine Corps has pursued developments with unmanned aerial ve¬ 
hicles to increase the force-multiplying capabilities that these enhanced, multispectral (Electro- 
Optical/Infrared and Synthetic Aperture Radar) systems bring to the fight. Newly emergent 
concepts for unmanned aerial systems (UASs) employment will continue to enhance and ex¬ 
tend the lethal and nonlethal capabilities of the Marine Air Ground Task Force (MAGTF). Joint 
Force Commanders will attain new levels of battlespace command and control and situational 
awareness by applying these newly emergent concepts. 

A Concept of Operations (CONOPS) for the United States Marine Corps Unmanned Aircraft 
Systems, Family of Systems (FoS) [2] was developed to provide fundamental guidance for 
initial concepts, tactics, techniques, and procedures (TTPs) in order to meet the need of persis¬ 
tent Intelligence, Surveillance, and Reconaissance (ISR) on today’s battlefield. As part of this 
CONOPS, three groups of unmanned aerial vehicles (UAVs) have been identified for employ¬ 
ment by the USMC. They are Group 1, Group 3, and Group 4 systems. 

Previous studies examined the interactions of UAVs amongst themselves in the context of target 
acquisition and engagement. The Navy is studying future applications and use of UAVs. This is 
highlighted by the Chief of Naval Operations (CNO) most recent Strategic Studies Group (SSG) 
XXVIII study on autonomous systems, entitled “The Unmanned Imperative.” Specifically, the 
SSG was tasked to show how unmanned and manned systems interact and to optimize the com- 
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mand and control structure to better integrate unmanned naval assets with manned systems. The 
SSG believes this integration will allow the Naval services to overcome the challenges of the 
21st century. In order to accomplish this, UAVs need to become more capable. Future use of 
UAVs will require them to be distributed globally, dispersed geographically, and have disag¬ 
gregated functionality. Global distribution allows the commander to gain regional knowledge 
through persistent ISR. Geographical dispersion provides for greater battlespace awareness. 
Disaggregated functionality refers to tailored functions and capabilities based on mission need. 

One of the critical capabilities of the UAV is autonomy. Autonomous systems can be thought 
of as possessing different “levels of autonomy” with some degree of human interaction com¬ 
bined with machine automation. Future CONOPS for all services will increasingly rely on this 
capability in some form, as many current UAVs, such as the Global Hawk, already operate with 
some degree of autonomy. A high level of automation will be required for future concepts that 
include large numbers of UAVs. In many large-scale operations where high numbers of UAVs 
may operate, the ability to self-coordinate may be needed due to the increased difficulty for 
operators to control multiple teams of UAVs. 

However, there is a pitfall if the number of UAVs and their uses are increased without viewing 
them as part of an overall distributed force. Many unseen and unexploited opportunities for 
the use of these systems may be missed. Thus, it is important that these systems be utilized 
in such a way as to maximize the integration of unmanned and manned systems. Studying the 
integration and future employment options among UAVs and the interdiction teams they are 
tasked to support is just a first step in this process. If ground interdiction teams are augmented 
with an organic tracking capability (the ability to track targets on their own) in the form of a 
man-portable UAV, the potential exists to more rapidly acquire and track a target, which would 
allow the surveyor UAV to be released from tracking and return to its primary mission of ISR 
and target acquisition. It is believed the natural result of this interaction would be an increase 
in the number of targets acquired and captured during interdiction missions. 

1.1.1 Overview of USMC UAV Family of Systems (FOS) 

The essence of the USMC FOS is to provide the MAGTF with dedicated operational capabilities 
focused on battlespace awareness and force application while enabling enhanced command and 
control (C2) throughout the range of diverse military operations. In order to accomplish this, 
the Marine Corps will need to acquire significant numbers of smaller UAVs that will be organic 
to the ground combat elements table of equipment. These smaller UAVs will become part of 


2 



UAS Category 

Max Gross 

Takeoff Weight 

Normal Operating 
Altitude (Ft) 

Airspeed 

Representative UAS 

Group 1 

< 20 pounds 

< 1200AGL 

<100 Kts 

Wasp, Raven B 

Group 2 

21-55 pounds 

< 3500 AGL 

< 250 Kts 

Scan Eaqle 

Group 3 

< 1320 pounds 

<18,000 MSL 

RQ-7 Shadow 

Group 4 

> 1320 pounds 

Any 

Airspeed 

RQ-9 Reaper 

Group 5 

> 18,000 MSL 

RQ-4 Global Hawk 


Table 1.1: Joint UAS Category 


the different “Groups” of UASs within the Marine Corps based on their capabilities. The group 
definitions have been categorized by the Joint UAS Center of Excellence based on weight, 
operating altitude, and airspeed (see Table 1.1). The greater these attributes, the higher the 
group number. Within the Marine Corps, Group 1 systems are micro and small UAVs, Group 3 
systems are small tactical UAS (STUAS), and Group 4 systems are Marine Corps tactical UASs 
(MCTUAS). The Marine Corps currently does not have a Group 2 UAS requirement. 

Group 1 systems include the RQ-1 IB Raven and Wasp. Wasp is organic to the infantry company 
and employed at the platoon or squad level. It is designed to be a small, rugged, man-portable 
system for employment at the tactical unit level for ISR missions. The Raven is organic to the 
infantry, artillery, armor, light reconnaissance, and amphibious assault battalions and employed 
at the company or battery level. It provides the company commander with enhanced situational 
awareness and observation of targets beyond the company’s line of sight (LOS). It is larger 
than Wasp; however, both systems utilize common control stations and thus operation of these 
systems is similar. These systems are designed for close-in reconnaissance, surveillance, and 
target acquisition in support of infantry squads, platoons, companies, and battalions. 

Group 3 systems are designed to be simple, transportable and rugged, yet capable of producing 
actionable intelligence for the maneuver commander. Group 3 systems will provide dedicated 
UAV support at the battalion level. Each system will have the following capabilities: electro- 
optical/infrared (EO/IR) sensors, integrated IR marker and laser range finder/designator, and 
communication relay packages. These systems are designed to be shipboard compatible with 
an unobstructed LOS operating radius of approximately fifty nautical miles (NM), and operate 
up to altitudes of 15,000 feet for extended periods of up to ten hours. Group 3 UAYs will be 
assigned to the Marine Unmanned Aerial Vehicle Squadrons (VMU) within the Marine Aircraft 
Wing (MAW). 

Group 4 missions are currently performed by a Group 3 system according to the definition. 
The future Group 4 system will be the MAGTF’s long-range, high-speed UAV with mission 
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sets that include C2, ISR, communication relay, target acquisition and marking, and precision 
strike. This system would be employed at the Marine Expeditionary Brigade (MEB) or Marine 
Expeditionary Force (MEF) levels. This platform may have future missions of performing the 
role of a cargo UAV or as an electronic warfare (EW) platform. 

1.1.2 Future Vision of the USMC UAV FOS 

The UAS FoS provides the Marine Air-Ground Task Force (MAGTF) and its subor¬ 
dinate units dedicated operational and tactical, interoperable, integrated and tailored 
Battlespace Awareness and Force Application capabilities while enabling enhanced 
Command, Control, and Communications throughout the range of military opera¬ 
tions. USMC UAV perform key roles in all Six Functions of Marine Aviation: 

Aerial Reconnaissance, Electronic Warfare (EW), Offensive Air Support (OAS), 
Assault Support, Anti-air Warfare (AAW), and Control of Aircraft and Missiles. 

[ 2 ] 


The above statement is a sign of the future employment of UAVs within the Marine Corps. 
UAVs will be expected to perform and succeed in the six functions of Marine Aviation. Cur¬ 
rently, with the exception of aerial reconnaissance, the six functions are performed by manned 
fixed and rotary wing aircraft. The vision is designed to not only save lives, but to provide 
for a high degree of interoperability among these systems and ground units to accomplish the 
mission. Some of these future concepts include Cargo UAS, Casualty Evacuation (CASE- 
VAC) UAS, Manned-Unmanned Teaming (MUT), Unmanned Combat Air System (UCAS), 
cross cueing (adding information about a target of interest), and intelligence fusion (combing 
all information about a target). 

Cargo UAS was envisioned as a need to reduce the number of convoys that are exposed to 
the improvised explosive device (IED) threat. By utilizing UASs for logistic support, potential 
IED attacks can be avoided and resupply can happen much more quickly, due to direct routing 
of the UAV and its speed. In a similar manner, UASs can be utilized to evacuate casualties 
from the combat zone and deliver them to field hospitals throughout the AO. MUT is the focus 
of this thesis and will be discussed in detail throughout. UCAS is envisioned to reduce the 
need to send manned aircraft into hostile areas. These UAVs will be able to deliver the same 
precision weapons as manned aircraft. They will also be smaller, lighter, and stealthier than 
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manned aircraft, thus reducing their radar cross-section and vulnerability to detection by enemy 
radar. This low observability translates into fewer fatalities in the combat zone. Cross cueing 
and intelligence fusion are concepts that are being practiced today but with the development 
of new technologies, these capabilities will become even more powerful. A real-time, fused 
intelligence picture will allow the analyst and warfighter to distinguish targets in cluttered, urban 
environments more easily, thereby increasing the likelihood of overall mission success. 

With many of these concepts in mind, we will examine a small part of the future requirements 
in this thesis. We will seek to examine MUT, specifically Group 1 and Group 3 interactions 
with a quick reaction force (QRF) or interdiction team, to aid in the development of a CONOPS 
for UAS employment as described in the next section. 


1.2 Problem Statement 

The Situational Awareness for Surveillance and Interdiction Operations (SASIO) software pro¬ 
vides an analysis tool and is designed to study mission characteristics and performance involv¬ 
ing surveillance assets such as UAVs in conjunction with interdiction assets such as ground- 
based QRF teams. We will examine different teaming strategies and coordination measures 
between searching and interdicting assets in order to study the effectiveness of the interdictor 
possessing an organic, tracker UAV. The team’s effectiveness is measured by the number of 
successful interdictions, i.e., detection, interception, and capture, of targets of interest. 

Currently, there is a one-way flow of information, in that a ground unit may request an airborne 
asset to assist with its mission, but they do not have direct control over that asset. It is either 
piloted remotely from the Tactical Operations Center (TOC) or by aircrew of a fixed or rotary 
wing aircraft. As UAVs become more prevalent on the battlefield, ground forces will have to 
increasingly rely on them for ISR as well as target marking, overwatch operations, and perhaps 
even enemy engagement. There exists a need within all services for new or better TTPs and 
CONOPS for effectively teaming these ground units with organic UAVs. 

The objective of this research is to quantify the benefit or penalty of an additional UAV asset in 
the context of the overall surveillance and interdiction operation using the SASIO model. 


1.3 Research Questions 

The objectives of this thesis research are guided by the following research questions: 
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• Is it better for an interdiction team to possess an organic “augmented tracking” capability 
in the context of time to intercept and number of targets captured? 

• What are the significant factors that produce teaming strategies that result in the greatest 
number of hostile targets captured? 


1.4 Literature Review 

As UAVs become integrated on the battlefield, it is increasingly important to operate them in 
a highly efficient manner. Many concepts apply to future UAV operations. One concept is the 
efficient routing of UAVs in the Area of Operations (AO). Efficient routing needs to take into 
account terrain, mission, search pattern, number of targets, and the network structure of the 
AO. This concept has been studied in [3], where the authors developed a decision support tool 
utilizing a dynamic program that provides efficient search routes for multiple UAVs searching 
for multiple targets on a known graph of nodes and arcs. This problem was extended to the 
study of path planning for multiple UAVs [4]. In this study, the authors addressed the problem of 
generating feasible routes for multiple UAVs in a target rich environment. An anytime algorithm 
was developed using particle swarm optimization that results in routes whose quality increases 
with an increase in the computation time. 

A second concept of UAV employment is that of sensing or target acquisition. Two significant 
factors that affect sensing are the probability that a target exists at a given location and the 
probability of the UAV seeing and identifying a target. One can use the number of hostile 
targets identified as a measure of effectiveness (MOE) for a given employment strategy. [5] 
investigated the value of predictive target assignment as a function of the number of unknown 
targets and number of UAVs. Target neutralization time (i.e., the time needed to neutralize 
targets) and total mission time (i.e., the time needed to destroy all targets) were the MOEs used. 
It was found that utilizing a prediction algorithm for target locations helped the cooperative 
UAV teams locate targets. As targets were located, they were stored in a database used by 
the UAV team to update target knowledge and thus update the prediction algorithm. As the 
target database increased, the predictions improved. In this study, the nominal density of targets 
within the AO will be known a priori and we will utilize different search algorithms as factors 
for our MOE. [6] studied the value of utilizing probabilistic information about reports of object 
detections and incorporated this information into a database that includes probabilities of an 
object’s existence as well as probabilities of its location. This aids in the discrimination of false 
and real objects. 
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A natural extension of UAV teaming is the concept of MUT. MUT is a concept that takes 
advantage of a UAV’s ability to detect targets and a ground team’s ability to interdict those 
targets. It is precisely this concept that we study in this thesis. An integer linear program 
was developed to optimize the employment and deployment of UAVs integrated with Special 
Forces [7]. The goal of this model was to assist commanders in determining suitable locations 
for mobile control centers (MCC) and ground control units (GCU) as well as optimizing the 
search areas for the UAVs. The model resulted in 50% more target detections than manual 
plans generated by experienced commanders. 

In order for success on the battlefield, UASs cannot be limited to operate in isolation. There 
must exist a level of coordination and integration with ground units. This problem was studied 
in an attempt to quantify the benefit of small, hand-launched UAVs as border patrol agents 
attempted to classify and capture illegal aliens at border crossing sites [8]. The MOEs used 
for this study were the number of illegal immigrants classified and captured and the number of 
smugglers classified and captured. This simulation study found there was an increase in correct 
classifications and captures with the aid of these UAVs. 

Utilizing similar methodologies as the works above, we determine the benefit or penalty to 
interdiction efforts by the addition of an organic UAV capability for a single interdiction team 
when used in conjunction with a surveyor UAV. We measure the number of targets captured for 
a given AO size and target set. Target locations are not be known a priori and we do not utilize a 
predictive algorithm for target locations. Specifically, we use the number of targets captured and 
the time to capture as our MOEs. These results then have the potential to affect future CONOPS 
for the integration of UAVs with ground combat elements. Results of our simulation contribute 
to the development of a decision support tool that can aid the Commander as he determines 
which targets to interdict based on which presents the highest priority. 

The remainder of this thesis is organized as follows. In Chapter II, presents the overall scenario 
to include a description of the problem domain. In this chapter, we also discuss constraints, 
assumptions and present the simulation model formulation. Chapter III discusses the exper¬ 
imental design used and presents a thorough discussion of factors, levels, response variables 
will also be presented. Chapter IV provides the analysis of the design and any insights gained 
from the simulation. Chapter V reports the conclusions and provide any recommendations that 
can be incorporated into future CONOPS as well as future model implementation. 
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CHAPTER 2: 
Model Formulation 


This chapter will discuss the SASIO simulation model and its operational context. We begin 
with a general model overview followed by a discussion of the three core components: the 
theater, the scenario, and the task force. Each component presents the various characteristics 
and any entities it possesses and discuss the motion models that govern the entities. 

2.1 Operational Context 

The simulation environment possesses an Area of Interest (AOI) in which QRF teams are lo¬ 
cated at a FOB with the mission of interdicting and capturing hostile targets. Vehicle mounted 
QRFs will be restricted to the existing road networks. Foot mobile QRFs are not restricted to 
the road networks and although terrain often dictates ease of travel, for this thesis, terrain will 
not be a factor. The QRF can travel either by foot or vehicle (e.g., HMMWV) to the target 
location. The QRF will remain at the FOB until it receives a mission. 

Airborne assets include a surveyor UAV that will perform ISR within the AOI. This UAV will 
utilize one of three specified search patterns to determine which provides the most target de¬ 
tections. Missions will be generated from reports sent to the FOB by the surveyor. After the 
report is generated surveyor can perform one of two missions. First, it can continue to search 
for additional targets. Alternatively, it can transition to a tracking mode, updating target action 
and location in the form of additional reports to the QRF. Once the QRF arrives at the target 
location, the surveyor will be released back to its search mission. 

The QRF has the ability to have a tracker UAV assigned to it as an organic asset. Faunched 
by the QRF when the surveyor issues a report, the tracker will proceed to the last known target 
location. If the surveyor is in tracking mode, the tracker will conduct a target handoff, once it 
has positively identified the target, releasing the surveyor to its search mission. 

2.2 Overview of the Model 

SASIO is an agent based simulation model written in the Java programming language. Agent 
based modeling is used to study the interactions of autonomous agents in complex systems. 
These models are used to simulate the actions of multiple agents in an attempt to predict or 
re-create complex phenomena. Monte Carlo techniques can be used to introduce randomness 
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to the model. SASIO defines both agent and object modeling. “Agents” refer to friendly forces 
and “objects” refer to either neutral or hostile targets. SASIO is used to define and construct, for 
example, multiple search methods for agents, agent planning and interdiction behaviors, object 
motion models, and their respective interactions with the environment. 

2.3 Theater Description 

Within SASIO the theater describes the region in which all entities interact. The theater for this 
thesis will consist of a range of AOIs from 10 x 10 km to 100 x 100 km. Each AOI consists of 
1 km x 1 km grid squares and represent any location where real-world operations are currently 
ongoing. The choice for this size grid square is consistent with standard unit of measure for 
charts, maps, and gridded reference graphics used by ground and air assets in the current theaters 
of operation. 

Each AOI can be represented by an undirected graph with a corresponding symmetric adjacency 
matrix, which defines all connections in the network. We will define three adjacency matrices 
as follows: 


Gi is a grid graph where C = 100 
G 2 is a grid graph where C = 2500 
G 3 is a grid graph where C = 10, 000 


where C denotes the number of cells within each AOI. Each graph has eight-point connectivity, 
meaning each cell, c, may be connected in the following manner, (NW, N, NE, E, SE, S, SW, W} 
Edge cells are connected in the same manner but with fewer adjacent cells. The adjacency ma¬ 
trix takes the following form: 


Adj Gi 



110 0 0 
1110 0 
0 1110 
0 0 111 
0 0 0 1 1 


, where ay 


1 if connected by eight pt. connectivity 
0, otherwise 


These AOIs can be thought of as abstractions or extensions of Camp Roberts in California (see 
Figure 2.1) where ongoing research and experimentation utili z e UAVs and ground assets to 
study operationally relevant problems such as proposed in this thesis. 
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Figure 2.1: Camp Roberts Gridded Reference Graphic 

2.4 Scenario Description 

The scenario describes the number, movement, and prediction models of the red forces, known 
as objects. Objects in this thesis can either be targets or neutrals. 

2.4.1 Object Placement Model 

For this thesis, we will consider three separate object sets. We will let N be the number of 
objects and N e {30, 75, 120}. The make up of the object set is a one to two ratio of hostile 
to neutral objects. For example, if N = 30, 10 objects will be targets and 20 will be neutral. 
Object placement is assumed independent which may result in more than one object being 
placed within a cell. Each object set will be placed within the AOI based on a probability map 
that is initialized at the start of the scenario with probabilities of an object being placed within a 
cell. We will define the probability using a uniform distribution across the entire theater in the 
following manner: 

Let the discrete random variable X= the location of an object, where X = {1,2,3,...,C}, with 
probability distribution: 
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P[X = c] = the probability that an object is located in cell c, then for uniform probabilities 


P[X = c ]= l - 

In order to generate the object locations, for each object we will generate a random number 
based on the number of cells in the AOI and then place the object at that cell location. For 
example, 


R~U(1,C) 

where recall that C is the number of cells in the AOI. This algorithm will continue until all 
objects have been assigned. Since all object locations are assumed independent, all assignments 
will follow the same uniform distribution. 

2.4.2 Object Motion Model 

In order to model object motion, we need to know the object’s current location, all possible fu¬ 
ture locations, and the time step utilized to measure target velocity. Object motions are modeled 
as a discrete time Markov chain with a time step equal to one minute, i.e., At = 1. A discrete 
time Markov chain is specified by its transition matrix and its initial distribution. The objects’ 
initial distribution, as described above, is given by 

P[X — c] — which can be written as P[X L ] = P\X 2 ] = P[X 3 \ - ... = P[X C ] = 

Target locations can be determined by starting with their initial location given by the target 
placement model. The theater adjacency matrix represents all possible future locations for a 
target in a given cell. The adjacency matrix is a undirected graph that depicts all possible target 
flow from cell to cell. When the future locations are replaced by the probability of an object 
transitioning to that cell, the resulting matrix becomes the theater transition matrix, P. In order 
to determine the transition probabilities we use the following algorithm. This algorithm defines 
all one time step transition probabilities. 

Ti t+1 =Pn t , where Tt i+1 is the probability of transitioning in one time step, P is the transition 
matrix, and tc* is the current location probability. This thesis will explore the Random Walk 


12 



motion model that can be generalized to different object motions. The object motions will 
be characterized by the transition probability possessed by the object and can range from a 
stationary target to a target that transitions at every time step. 

Since the theater is represented by an undirected graph, the adjacency matrix is symmetric 
meaning if i is connected to j, then j is connected to i. A row of an adjacency matrix represents 
all cells adjacent to a given cell, i.e, if a cell j is adjacent to cell i, the (i, j)— entry in the 
adjacency matrix is one. The sum of this row is called the outdegree, which is defined as the 
sum of all arc tailpoints that emanate from that cell and connect to another cell. Formally 


c 

outdegreej = ^ a i: j 
j =i 


Non-Uniform Random Walk Motion Model 

In the non-uniform random walk model, the probability of transitioning to any adjacent cell is 
not described by a uniform distribution. Instead, the current location can be described as being 
self-important, meaning the target either wants to quickly leave the cell in which it is located or 
it would prefer to stay at the current location. The first case can be described as a fast random 
walk and the second as a slow random walk. In either case, this self-importance imparts a 7T ht , 
the self-transition probability for the current cell, that is different from its adjacent cells which 
is the cause of the non-uniformity. In order to find the transition probabilities for the remaining 
cells we need to perform the following calculation 




1 


outdegree — 1 


, j j e Adj 


Gi 


7i id = 0 if j (£ Adj Gi 


Row entries for each c\ are determined in this manner. The resulting matrix is the transition 
matrix, P. 

To take this argument one step further, since is a continuous factor, we see that as x,,,; 
approaches the limit of 1, the transition probability to other cells approaches zero and thus we 
have the special case of the non-uniform random walk that describes a stationary target. 
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Figure 2.2: Lawnmower Search Pattern 


It can also be shown that uniform random walk is a special case of the non-uniform random 
walk. In this case the transition probabilities are derived by the following equation 

1 

1,3 outdegree 

that shows the object is equally likely to go to all adjacent cells and therefore has a transition 
matrix that is uniform for each discrete time step. Uniform walk will not be explicitly studied, 
since it falls within the continuum defined by the fast and slow random walk cases. 

2.5 Task Force Description 

The final part of SASIO is the taskforce that describes the blue force assets to include the 
surveyor, QRF, and the tracker UAV organic to the QRF. The taskforce describes all blue force 
agent capabilities and motion models. 

2.5.1 Surveyor UAV 

Search Pattern 

The surveyor UAV represents a Group 3 system and will perform ISR in order to detect hostile 
targets within the AOI. It will fly one of three search patterns as directed by the operator. The 
possible patterns are lawnmower, spiral, and random walk (See Figure 2.2). 

Identification 

The surveyor is modeled with imperfect sensing capabilities and is the only Blue Force as¬ 
set that must address this issue. The main characteristics of sensing are p, the false negative 
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identification probability and 7 , the false positive identification probability. When the surveyor 
makes a detection there are four possible probabilities for target detection based on the sensing 
characteristics. These cases are (1) the probability of a correct negative identification (correctly 
observing no target present), I- 7 , ( 2 ) the probability of a false positive identification (incor¬ 
rectly observing a target when none is present), 7 , (3) the probability of a missed detection 
(incorrectly observing no target when one is present), p, and (4) the probability of correctly 
detecting a target, 1-p. These probabilities can be expressed using the following derivation: 

Let X c be a Bernoulli random variable describing target presence as defined below 


*c = 



no target present 
at least one target present 


Let Y c be a Bernoulli random variable describing target detection as defined below 


We can then say, 



target not detected 
target detected 


l-'y = P[Y C = 0\X C = 0] 
7 = P[Y C = 1\X C = 0 ] 
p = P [Y c = 0\X C = 1] 
1-p = P[Y c =l\X c = l] 


Surveyor Tracking Behavior 

The surveyor is modeled as having one of two different capabilities. In the first case, the sur¬ 
veyor does not have a tracking capability. It executes its assigned search pattern, as per section 
2.5.1 and upon making a detection issues a report containing the time and location of the detec¬ 
tion. Since it has no tracking capability, the surveyor will continue its assigned search pattern 
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Figure 2.3: Surveyor with Tracking Capability Finite State Machine 


and make additional reports as necessary. In the second case, the surveyor does possess a track¬ 
ing capability. Upon detection of a target, it will issue a report but now transition into a tracking 
mode. The surveyor will continue to track the target until it released by the QRF or the tracking 
UAV at which time it will return to its search mission. 

2.5.2 Interdiction Team (QRF) 

Upon initialization of the simulation, the QRF is in a stationary mode, placed at the FOB loca¬ 
tion, and is listening for reports from the surveyor. The interdiction team will have two modes 
of transit: foot mobile and vehicle mounted. At each time step, the QRF listens for a report 
and if one is received it will begin its transit to the requested or goal location. Once the goal 
location is reached, the QRF will clear the target, and check for any unserviced reports from the 
surveyor. If there are unserviced reports, the QRF will transit to the new goal location. If there 
are no new reports, the QRF will transit back to the FOB. Once at the FOB, the QRF returns 
to stationary mode and begins listening for new unserviced reports. It is worth noting that the 
interdiction team brings two factors with three levels each to the simulation. The first is transit 
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Figure 2.4: Interdiction Team Finite State Machine 


time, which is a function of the mode of travel and the second is clear time, which will take on 
one of three values which will represent a delay at the goal location. This process will continue 
until either all targets are interdicted and cleared or the simulation ends. 

The QRF may also have the added capability of an organic tracking UAV. The QRF model 
would now need to launch the tracker at the specified time or distance from the goal location. 
Also, release of the surveyor would be initiated by the tracker UAV once it is has received 
target handoff from the surveyor. Now when the QRF reaches the goal location, it will assume 
tracking responsibilities and release the tracker for any additional unserviced reports. 

2.5.3 Tracker UAV 

The tracker UAV represent a Group 1 system and will be initialized in a stationary mode co¬ 
located with the QRF and awaits tasking from the QRF. The tracker UAV represents a portable, 
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Figure 2.5: Tracker UAV Finite State Machine 


hand-launched asset organic to the QRF. As the QRF transits to the goal location, it will launch 
the tracker at one of three specified distances from the goal location. These distances are one, 
three, and five cells from the goal location. The tracker velocity is 60 and 180 kilometers per 
hour which corresponds to approximately 35 and 100 knots. It is also assumed to have perfect 
sensing capability. Upon receipt of tasking, the tracker is launched by the QRF to begin its 
transit to the goal location. After receiving the target handoff from the surveyor, the tracker will 
release the surveyor back to its search mission and assume tracking responsibilities until arrival 
of the QRF. As the QRF arrives at the goal location, it will release the tracker for any additional 
unserviced reports if they exist. If there are none, then the tracker returns to the QRF. 
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CHAPTER 3: 
Experimental Design 


We use experimental design as the basis for determining the significant factors for the presented 
SASIO simulation model. Experimental design allows for estimating the effects of the input 
variables simultaneously where variation of the factors is present. Screening experiments allow 
the experimenter to vary factors in such a way that the most significant factors can be identified 
with respect to the response variables of interest, with as little experimental effort as possi¬ 
ble. The SASIO software encapsulating the model is designed to utilize design of experiments 
(DOX) to study the aspects of UAVs’ surveillance characteristics in conjunction with ground 
based interdiction teams with the overall goal of increasing the number of targets captured. 

The model created for this thesis has several response variables of interest. The primary re¬ 
sponse variable is the number of targets captured. This is the metric by which we measure 
the benefit or penalty of the different teaming strategies. The second to be quantified is the 
percentage of time the surveyor UAV actually spends performing the search mission. 

The remainder of this chapter introduces the response variables of interest as well as the factors 
to be used. A thorough explanation of the particular experimental design used will also be 
discussed. Finally, this chapter concludes with the analysis of the output from the simulation 
runs. 


3.1 Response Variables 

There are many factors and levels that will vary during this experiment. The model will be 
used to simulate the unpredictability of all entities in the AOI and thus will provide insights 
that demonstrate which factors have a significant effect on the response variables. We have two 
response variables for this simulation model and their descriptions follow below. 

3.1.1 Percentage of Targets Cleared 

This is the primary response variable that any commander will wish to know. The percent 
targets captured directly relates to how efficiently assets perform for a given teaming strategy. 
The higher this number, the more successful the teaming strategy. Insights can be gained on 
which QRF mobility type is preferred and if it is worth the effort to launch the tracker based on 
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Factor 

Levels 

Type 

Description 

Team Type 

Surveyor 

Categorical 

UAV capabilities that 
are available to the QRF 

S urvey or/Tracking 

Surveyor with Tracker 

Search Pattern 

Random Walk 

Categorical 

patterns to be flown by 
Surveyor UAV only 

Lawnmower 

Spiral 

Tracker Launch 

[1,3,5] 

Nominal 

# cells from goal location 

Interdictor Transit Time 

[15,2,1] 

Nominal 

# time steps to traverse 1 cell 

Tracker Speed 

[1,3] 

Nominal 

# cells traversed / time step 

Surveyor Gamma ( 7 ) 

[0, 0.9] 

Nominal 

Constraint 

7 + p < 0.9 

Surveyor Rho (p) 

[0, 0.9] 

Nominal 

Search Area 

[100, 2500, 10000] 

Nominal 

# cells, C in the AOI 

Interdictor Clear Time 

[ 1 , 11 , 21 ] 

Nominal 

# time steps 

Number of Objects 

[30, 75, 120] 

Categorical 

1:2 target to neutral ratio 

Object Motion 

Slow Random Walk 

Nominal 

Dependent on self 
transition probabilities, 77 ^ 

Fast Random Walk 

Nominal 


Table 3.1: Factors and Levels 


QRF distance to the target. This response will be obtained directly from the interdictor cleared 
list stored within SASIO. 


3.1.2 Percentage of Time Surveyor UAV Performs Search 

This secondary response relates to the percent targets cleared. Of interest is how augmented 
tracking affects this response when the surveyor possesses a tracking capability. This response 
will be obtained by subtracting the surveyors tracking time from the total simulation time and 
then dividing by total simulation time. The purpose behind the different teaming strategies is to 
identify the strategy that results in the highest percent of targets captured per simulation run. 


3.2 Factors and Levels 

The following factors and levels to be investigated in this thesis are listed in Table 3.1. 

Each factor represents a characteristic of the entities in the simulation and represents a particular 
value the factor can take during the course of the mission. Each factor and its expected impact 
to the results follow. 
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3.2.1 Team Type 

Description 

The Team Type is the main focus of this thesis. Team Type is the ultimate decision a comman¬ 
der will need to make in order to achieve his objectives. The first strategy utilizes a surveyor 
UAV only with no tracking capability. The second strategy represents a surveyor UAV that has 
tracking capabilities. The third strategy will utilize the surveyor but it will be augmented with 
an additional tracking UAV, organic to the QRF. This factor can take on three discrete levels and 
is therefore categorical. 

Predicted Impact 

We predict the augmented tracking provided by the tracker UAV will allow the surveyor to 
spend more time in its primary mission of reconnaissance. As a result, more targets should be 
identified and captured during the course of the simulation. Thus, an increase in the primary 
and secondary response is expected. 

3.2.2 Search Pattern 

Description 

Search pattern is also a categorical variable and will have three levels: lawnmower, spiral out, 
and random walk. With the exception of the random walk, each represents a predefined set of 
waypoints for the surveyor UAV to travel as it searches for targets. The search patterns will not 
be affected by intelligence related to target locations. UAVs will fly the specific pattern until 
a target is detected, at which time a report will be issued and the surveyor will either track the 
target until released by the QRF or it will continue to search depending on its capabilities and 
the current strategy. 

Predicted Impact 

The search patterns we will use make a difference if there are a large number of targets randomly 
clustered together where the UAV has the ability to find one right after another. If the targets 
are placed randomly far apart, then we do not anticipate the search pattern having much of an 
effect on the percent targets captured. 

3.2.3 Tracker Launch 

Description 

The launch of the tracker UAV is modeled using three thresholds, representing the number of 
cells from the target when the tracker is launched. The three values are one, three, and five cells. 
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Predicted Impact 

This factor when combined with the tracker speed will effect the amount of time the surveyor 
will spend conducting its primary mission of reconnaissance. The earlier the tracker can be 
launched and the faster it can arrive at the goal location, the sooner it can relieve the surveyor 
and thus return to its search mission. 

3.2.4 Interdictor Transit Time 

Description 

These values will represent the number of time steps required to traverse from one cell to an 
adjacent cell. Each value simulates different modes of travel from a foot mobile QRF to a 
vehicle mounted QRF traveling on a road network with velocities ranging from 4kph to 60kph. 

Predicted Impact 

As the velocity of the QRF decreases more time will be required to transit to a goal location. 
This translates into more time the surveyor has to remain on station before being released by 
either the QRF or the tracker, which we predict will negatively effect the number of targets 
identified and captured. 

3.2.5 Tracker Speed 

Description 

This factor represents the different velocities the tracker UAV may fly. We assume perfect 
sensing for the tracker so speed will not affect tracker 7 and p. It has two levels, one and three, 
which represent the number of time steps required to traverse from one cell to the next cell en 
route to the target location. These levels are equivalent to 60 and 180kph. 

Predicted Impact 

Assuming surveyor will be released by the tracker, the quicker the tracker can arrive at the goal 
location to release the surveyor, the sooner the surveyor can return to its mission. This should 
lead to an increase in the percent targets captured and the total number of targets identified. 

3.2.6 Sensor Characteristics 

Description 

Two continuous factors are used to represent the imperfect sensing capabilities of the surveyor 
UAV. 7 (gamma) is the probability of having a false positive, which is identifying a target as 
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hostile when it is not. p (rho) is the probability of a false negative, which is classifying a target 
as friendly when it is in fact hostile. 

Predicted Impact 

While both 7 and p can have a significant effect, 7 is a greater concern due to the misclassifica- 
tion of a neutral as a target. The higher this misclassification rate, the more time the QRF will 
spend clearing objects that are not targets and thus as 7 increases for the surveyor, we would 
expect to see the percent targets captured decrease. 

3.2.7 Search Area 

Description 

The search area is a continuous factor based on AOIs composed of 1 km x 1 km grid squares. 
Varying this factor will also help quantify the benefit or penalty of the QRF having an organic 
UAV tracking capability. 

Predicted Impact 

An increase in search area should make it more difficult to locate targets. Therefore, we would 
expect to see a decrease in the percent targets captured as the size of the search area is increased. 

3.2.8 Clear Time 

Description 

This factor represents the amount of time it takes for the QRF to actually interdict and capture 
a target. This factor takes on three levels as shown in Figure 3.2. 

Predicted Impact 

We expect that as the clearing time is increased fewer additional targets found by the surveyor 
will be interdicted and cleared by the QRF. 

3.2.9 Number of Objects 

Description 

The number of objects in the AOI will take on three distinct values. The objects will be ran¬ 
domly placed throughout the AOI and will have no affect on the search pattern used by the 
surveyor or mode of travel of the QRF. The particular level represents a 1:2 ratio of targets to 
neutrals in the AOI. For example, if the number of objects equals 30, there are 10 targets and 20 
neutrals in that target set. 
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Predicted Impact 

As this factor is varied with different AOI sizes, we expect to see significant variation to the 
number of targets identified and therefore the percent targets captured. 


3.3 Experimental Design 

The choice of experimental design was developed to accommodate eleven input factors. This 
experiment constitutes a mixed-level design, for which the design matrix was created using the 
D-optimality criterion that results in a nearly orthogonal design. The result is nearly orthogonal 
because some correlation exists among the columns of the information matrix [X'X\. A D- 
optimal design attempts to minimize the variance of the model regression coefficients [9]. 

Factor screening is the process of systematically varying input factors in order to identify the 
factors that produce a significant change in the response variables. The screening experiment 
is used to estimate the magnitude and direction of individual factor effects as well as factor in¬ 
teraction effects on the response variable. As previously mentioned, we are using a /J-optimal 
design for the screening experiment in an attempt to minimize the variance of the model regres¬ 
sion coefficients. The D-optimal design minimizes the volume of the joint confidence region 
on the vector of regression coefficients by minimizing | (X'X)~ 1 1. which equates to minimizing 
the uncertainty of the regression coefficients. 

Traditional screening experiments are conducted using only two levels of the factors, a low 
level and a high level. Center points are added to the design to check for non-linearity within 
the model and to reduce the variance in the center of the design space. The resulting design 
matrix is read into the SASIO model and the simulation conducted using the actual values as 
found in Table 3.1. The design matrix developed in JMP 8.0.1 [10] resulted in 102 design points 
including six center points. 

In order to analyze the data, the actual values in Table 3.1 must be transformed to coded values 
that strips the units from any level and turns the result into a level that is dimensionless. This 
allows the magnitude of the model coefficients to be directly comparable and the effect of 
changing each design factor over a single unit is easily measured. Coded variables are effective 
for determining the relative size of factor effects and allow the experimenter to see the relative 
importance of the design factors. Output data is then analyzed in JMP but with the factor 
values converted to coded units of -1, 0, and 1 to represent the low, center point, and high 
levels respectively. There were some cases where we chose a third level that was not the exact 
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Std Dev 



Figure 3.1: Asymptotic Standard Deviation 


center point. This was done to more accurately represent the level as a real world value for the 
particular level. Actual values were converted to coded units using the following formula: 


coded value 


Actual value — (Actual Low + Actual High )/2 
(Actual H ig h - Actual Low )/2 


3.4 Methods of Analysis 

3.4.1 Asymptotic Variance 

Through a study of the asymptotic variance, we determined 60 replications of each design point 
was adequate to gain insights from the simulation model, see Figure 3.1. We ran the model 
for one design point multiple times while varying the number of replications until the standard 
deviation of both response varibles stabilized. This occured at approximately 60 replications. 
We performed this analysis in order to determine the minimum number of runs per design point 
while being able to gain insights from the simulation with the goal of running the simulation on 
a tactical laptop in the field. The simulation can run from less than an hour to approximately 
six hours depending on the number of design points. 
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3.4.2 Multivariate Linear Regression 

Multivariate linear regression can be used to determine what factors in the screening experiment 
have a significant effect on the response. In a linear regression model the response variable, y 
is related to predictor variables, x through the following relationship 


y — A) + 0\X 1 + 02X2 + 01, 2 X 1 X 2 + • • • + 0n x n + £ 

The standard multivariate linear regression model tests the following hypothesis: 

H 0 : 0o = 0 1 x x = 0i, 2 X 1,2 = 0n x n = 0, where nis the number of coefficients 
Hi : at least one coefficient ^ 0 

3.4.3 Logistic Regression 

Logistic regression is used for the prediction of the probability of the occurrence of an event. 
The event we are interested in is the probability of a target being cleared. The number of targets 
cleared is a Bernoulli random variable that can take on the values of zero or one. Either the 
target is cleared or it is not. When a response variable is binary, the resulting shape of the 
response function is nonlinear [11]. This nonlinear function takes the form 

logit = 

An examination of this function shows that it is easily linearized by using the logit transforma¬ 
tion defined below as 

77 = In ( 13 ^T)) ’ w * lcrc r /’ s l ' lc linear predictor of the response variable y 

By employing this transformation we are able to linearize the response variable and perform 
standard multivariate linear regression. 

In order to determine the primary response variable, the mean number of targets was determined 
for each design point and then divided by the number of targets for that design point to attain the 
percent targets cleared. The resulting percent was transformed via the logit function to ensure 
the response variable values remained between zero and one and is shown below 

logit(percent targets cleared) =ln ( , percent f toget ; .cleared ) =xj3+e 

° vr ^ 0 7 V 1 -percent targets cleared J 1 
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CHAPTER 4: 
Numerical Analysis 


As mentioned in Chapter 3, screening experiments were performed to determine the significant 
factors that affect the two response variables: (1) percentage of targets cleared, and (2) the 
percentage of time the surveyor UAV performs its primary mission of search. 

The first part of the analysis performed was to determine the effect of Tracker Launch and 
Tracker Speed only for the Team Type of Surveyor with Tracker. To accomplish this, we broke 
down Surveyor with Tracker into five separate factors each representing the high-low combi¬ 
nations and centerpoints of Tracker Launch and Tracker Speed. This was done to isolate the 
effects and because these effects are not present for the Surveyor only and Surveyor/Tracking 
Team Types. Table 4.1 shows how we broke down Surveyor with Tracker. 

The analysis was performed omitting Tracker Launch and Tracker Speed. Performing the anal¬ 
ysis in this way allowed for effects of Tracker Launch and Tracker Speed to be preserved for 
Surveyor with Tracker while at the same time being removed from the analysis in order to en¬ 
sure those factors do not impact the Team Types where the Tracker is not part of the simulation 
model. The remainder of this chapter will discuss the results of the screening experiments. 

4.1 Percentage of Targets Cleared 

The initial regression analysis showed eight main and interaction effects were significant. Fur¬ 
ther analysis showed that an equally good model, in terms of R 2 , could be achieved with only 
five main and interaction effects. The results are summarized in Figure 4.1. 


Team Type 

Tracker Launch Level 

Tracker Speed Level 

Surveyor with Tracker A 

1 

1 

Surveyor with Tracker B 

-1 

-1 

Surveyor with Tracker C 

1 

-1 

Surveyor with Tracker D 

-1 

1 

Surveyor with Tracker E 

0 

0 


Table 4.1: New Team Type-Surveyor with Tracker A-E 
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▼ & Response LOGIT Percent Mean Tgts Cleared 
▼ Whole Model 


▼ Actual by Predicted Plot 



Predicted P<.0001 RSq=0.91 
RMSE=0.9772 


▼ Summary of Fit 

RSquare 0.914576 

RSquare Adj 0.906034 

Root Mean Square Error 0.977221 

Mean of Response -4.21499 

Observations (or Sum Wgts) 100 

▼ Analysis of Variance 


Sum of 

Source DF Squares Mean Square F Ratio 

Model 9 920.1744 102.242 107.0637 

Error 90 85.9464 0.955 Prob > F 

C. Total 99 1006.1208 <.0001* 

Figure 4.1: Linear Regression model of LOGIT transformation of Percent Targets Cleared 


Figure 4.2 lists the parameter estimates in order of lowest to highest p- value. The regressor with 
the largest coefficient is Search Area. As the search area is increased, a negative response in 
the percent of targets cleared is anticipated. Since our analysis was conducted using coded vari¬ 
ables, we can obtain the odds ratio for a single unit increase in search area to be e~ 256 = 0.077, 
not taking into account the effects of the other factors. This odds ratio can be interpreted as the 
estimated increase or decrease in the Percentage of Targets Cleared [11] per unit increase on 
Search Area. Therefore as Search Area increases from its mid level to high level, the odds of 
success is approximately 8 out of 100. This model also shows that utilizing a team type com¬ 
prising only a Surveyor asset with no tracking capability produces the least desireable results 
in terms of the response. The corresponding odds ratio is e -2 ' 3 ' = 0.09, which equates to a 
reduction in the odds of success to 9 out of 100. A similar argument can be made for Interdictor 
Transit Time and False Negative Probability of detection. 

The interaction of Interdictor Transit Time and Number of Objects is worth examining even 
though its effect is relatively small. Figure 4.3 shows that with a large number of objects, 
increasing the interdictor transit time still does not improve the percentage of targets cleared. 
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▼ Sorted Parameter Estimates 
Term 

Search Area 

New Team Type[Surveyor) 

Interdictor Transit Time 
False Neg Prob 

New Team Type(Surveyor with Tracker B] 
Interdictor Transit Time*Number of Objects 
New Team Type[Surveyor with Tracker D] 
New Team Type[Surveyor with Tracker C] 
New Team Type[Surveyor with Tracker A] 


Estimate 

Std Error 

-2.566751 

0.100065 

-2.371272 

0.184337 

-0.744327 

0.101361 

-0.715261 

0.106146 

0.9453771 

0.309938 

-0.274924 

0.100843 

0.7168729 

0.311846 

0.3268827 

0.309746 

0.0389965 

0.310307 


t Ratio 



Prob>lt| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<.0001* 

0.0030* 

0.0077* 

0.0238* 

0.2941 

0.9003 


Figure 4.2: Sorted Parameter Estimates of LOGIT Transformation of Percent Targets Cleared in Order of Signifi¬ 
cance 
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Figure 4.3: Interaction Profile showing the change in response as Search Area size interacts with New Team Type 


The QRF is simply overwhelmed by the number of objects to interdict. This result supports the 
argument for more QRF or patrols if a target rich environment is anticipated. 

Tukeys Least Squares Means Differences allows us to perform a multiple comparison of the 
means to determine if there is a significant difference among any pair of factors. Tukeys method 
then groups the mean responses that are not statistically different as seen in the crossletter report 
(Figure 4.4). Also shown is a plot showing the difference in the means between Team Types 
with a tracking capability and those without. We can see a significantly lower mean response for 
Surveyor when compared with Surveyor/Tracking and Surveyor with Tracking. This supports 
our conclusion that possessing a tracking capability will positively impact the response. 
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Least 

Level Sq Mean 

Surveyor with Tracker B A -2.742302 

Surveyor with Tracker D A -2.970806 

Surveyor/Tracking A -3.344536 

Surveyor with Tracker C A -3.360797 

Surveyor with Tracker A A -3.648683 

Surveyor B -6.058951 

Levels not connected by same letter are significantly 
different. 

r LS Means Plot 



Figure 4.4: Tukeys Crossletter Report and Plot of the Least Squares Means showing New Team Type Grouped by 
Similar Means 


4.1.1 Percentage of Targets Captured with Tracking Capability 

After establishing that possessing a tracking capability is better than no tracking capability, we 
were interested in the significant effects when the Surveyor Team Type was not included in 
the model. For this analysis, all Surveyor data was removed. Initial results of the stepwise 
regression produced a good fit with 17 main and interaction effects. Further analysis illustrates 
that seven main and interaction effects is also satisfactory. Results are shown in Figures 4.5 and 
4.6. 

Figure 4.6 lists the significant factors from most to least influential for this model. Search Area 
has the greatest effect on the response. The odds ratio is e~ 2 ' 29 = 0.10, which equates to a 
reduction in the odds of success to 10 out of 100 per unit increase of Search Area, which is very 
close that of the model with Team Type Surveyor included. 

Interdictor Transit Time is the next factor in the list. This was an anticipated result as was 
the effect of the surveyor’s sensor characteristics. Increasing the False Negative Probability, 
p, reduces the percentage of targets captured due to the increase in targets being misclassified 
as neutrals. An increase in the Number of Objects, coupled with other factors, increases the 
response simply because as the number of objects increases so does the number of targets. 
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▼ 5 Response LOGIT Percent Mean Tgts Cleared 
▼ Whole Model 


▼ Actual by Predicted Plot 



Predicted Pc.0001 RSq=0.91 
RMSE=0.8291 


▼ Summary of Fit 

RSquare 0.913089 

RSquare Adj 0.901304 

Root Mean Square Error 0.829083 

Mean of Response -3.29737 

Observations (or Sum Wgts) 68 

▼ Analysis of Variance 


Source 

DF 

Sum of 
Squares 

Mean Square 

F Ratio 

Model 

8 

426.07409 

53.2593 

77.4817 

Error 

59 

40.55533 

0.6874 

Prob > F 

C. Total 

67 

466.62942 


<.0001* 


Figure 4.5: Linear Regression Model of LOGIT Transformation of Percent Targets Captured 


▼ Sorted Parameter Estimates 
Term 

Search Area 

Interdictor Transit Time 
False Neg Prob 

Search Area'Number of Objects 
False Neg Prob'Number of Objects 
Search Pattern! Random Walk)*False Neg Prob 
Search Pattern! Lawnmowerj'False Neg Prob 
False Pos Prob 


Estimate 

Std Error 

-2.296026 

0.106584 

-0.839092 

0.106551 

-0.786552 

0.128978 

0.4054962 

0.114252 

0.3097635 

0.108559 

-0.400553 

0.150251 

0.3800016 

0.158974 

-0.292024 

0.129502 


t Ratio 



Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

0.0008* 

0.0060* 

0.0099* 

0 . 0200 * 

0.0279* 


Figure 4.6: Sorted Parameter Estimates of LOGIT Transformation of Percent Targets Cleared in Order of Signifi¬ 
cance 


Of greater interest is the lack of any effect from either Tracker Launch or Tracker Speed. In 
neither model do these factors have an effect. Tracker capabilities were modeled using the 
Raven UAV, the USMC’s Group IUAV. Further study is required to determine if a Tracker with 
greater capabilities would have a greater effect on the response variable. 


4.2 Percentage of Time Surveyor UAV Performs Search 

Performing an initial analysis of search time yields a model with 10 main and interaction effects. 
Further analysis yields a model with only four main and interaction effects and R 2 of 0.92 and 
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T & Response LOGIT Percent Search Time 
▼ Whole Model 
▼ Actual by Predicted Plot 



LOCIT Percent Search Time Predicted 
P<.0001 RSq = 0.92 RMSE= 1.2791 


▼ Summary of Fit 


RSquare 

0.924418 


RSquare Adj 

0.913993 


Root Mean Square Error 1.279075 


Mean of Response 

4.088854 


Observations (or Sum Wgts) 

100 


Analysis of Variance 


Sum of 



Source DF 

Squares 

Mean Square 

F Ratio 

Model 12 

1740.8602 

145.072 

88.6728 

Error 87 

142.3349 

1.636 

Prob > F 

C. Total 99 

1883.1951 


<.0001* 


Figure 4.7: Linear Regression Model of LOGIT transformation of Percent Time Search 


▼ Sorted Parameter Estimates 
Term 

New Team TypefSurveyor] 

Search Area 

New Team Type[Surveyor)*Search Area 
False Neg Prob 

New Team TypefSurveyor with Tracker C] 

New Team TypefSurveyor with Tracker B] 

New Team TypefSurveyor with Tracker D] 

New Team TypefSurveyor with Tracker A]*Search Area 
New Team TypefSurveyor with Tracker B]*Search Area 
New Team TypefSurveyor with Tracker A) 

New Team TypefSurveyor with Tracker D]*Search Area 
New Team TypefSurveyor with Tracker C]*Search Area 


Estimate 

Std Error 

6.3044172 

0.241679 

1.7617465 

0.162107 

-1.88394 

0.245388 

0.9745043 

0.143901 

-2.037017 

0.414664 

-1.340732 

0.405956 

-1.133776 

0.403812 

0.527073 

0.414751 

0.4872861 

0.409871 

-0.330549 

0.414664 

0.2000334 

0.409376 

0.196679 

0.414881 


t Ratio 



Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

0.0014* 

0.0062* 

0.2072 

0.2377 

0.4275 

0.6263 

0.6366 


Figure 4.8: Sorted Parameter Estimates of LOGIT transformation of Percent Time Search in order of significance 


adjusted R 2 of 0.91. The results are shown in Figure 4.7. 

The results shown in Figures 4.7 and 4.8 show that we would reject the null hypothesis in favor 
of the alternative at any reasonable confidence level. 

When all Team Types are included in the model, the predictive power of this model can be 
skewed by the Surveyor team type as shown in Figure 4.8. This is due to Surveyor having no 
tracking capability, which therefore ensures that it will spend the entire duration of the mission 
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T [r Response LOGIT Percent Search Time 
▼ Whole Model 
▼ Actual by Predicted Plot 



LOGIT Percent Search Time Predicted 
Pc.OOOl RSq = 0.92 RMSE=0.8185 


▼ Summary of Fit 


RSquare 

0.92212 


RSquare Adj 

0.915839 


Root Mean Square Error 0.818547 


Mean of Response 

1.427759 


Observations (or Sum Wgts) 

68 


Analysis of Variance 


Sum of 



Source DF 

Squares 

Mean Square 

F Ratio 

Model 5 

491.85541 

98.3711 

146.8185 

Error 62 

41.54114 

0.6700 

Prob > F 

C. Total 67 

533.39655 


<.0001* 


Figure 4.9: Linear Regression Model of LOGIT Percent Time Search 


searching. Of additional interest is how Percent Search Time responds when the team types 
include a tracking capability, which is described in the following section. 


4.2.1 Surveyor Search Time Percentage with a Tracking Capability 

The initial analysis excluding the Surveyor only Team Type yielded a model with eight main 
and interaction effects. Further analysis showed that a model with only five factors produced an 
R 2 of 0.92 and an adjusted R 2 of 0.91. These results are summarized in Figure 4.9 and 4.10. 

The sorted parameter estimates in this case show that the greatest effect is due to the Search 
Area. The odds ratio is e ~ 2 ' 34 = 0.09 which is a reduction in the odds of success to 9 out of 
100 for each unit increase in Search Area size. Next, the False Negative Probability increases 
the search time due to an increase of the misclassification rate of targets as neutrals. This leads 
to Surveyor not tracking and thus increases the search time at the expense of targets captured. 
The next factor with the greatest effect is the Number of Objects. As the Number of Objects 
increases, the percentage of time the Surveyor searches will decrease due to the tracking time 
involved with each additional target found. False Positive Probability has the opposite effect 
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▼ Sorted Parameter Estimates 
Term 

Search Area 

False Neg Prob 

Number of Objects 

Interdictor Transit Time’Search Area 

Flase Pos Prob'False Neg Prob 


Estimate Std Error 

2.3492065 0.102521 

1.7504826 0.127077 

-0.707536 0.102929 

0.6905477 0.102882 

0.5635082 0.124617 


t Ratio 


22.91 




13.77 



1 

-6.87 




6.71 

4.52 



kiL 


Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 


Figure 4.10: Sorted Parameter Estimates of LOGIT Percent Time Search in order of significance 
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Figure 4.11: Interaction Plots of Search without Surveyor Team Type 


of False Negative Probability. As the False Positive Probability rate increases, Surveyor spends 
less time searching and more tracking neutral targets. 

The interaction of Interdictor Transit Time and Search Area provides an interesting insight to 
this model as depicted in Figure 4.11. 

An analysis of the interactions without the Surveyor Team Type is shown in Figure 4.11. This 
plot shows that while Search Area is at its high level (1), as the Interdictor Transit Time de¬ 
creases (faster interdictor speed) the percentage search time decreases. At an Interdictor Transit 
Time of 15kph, its high level, the Search Area is so large that the Surveyor spends a longer 
time tracking than searching while the QRF is en route to the target location. At the low level 
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Search Pattern 

Lawnmower 

Tracker Launch 

3 

Interdictor Transit Time 

1 

Tracker Speed 

3 

False Positive Probability 

0.3 

False Negative Probability 

0.3 

Clear Time 

20 

Number of Objects 

120 

Object Motion 

SlowRW 


Table 4.2: Factor Levels used for Sensitivity Analysis 


of Search Area, as Interdictor Transit Time decreases, the percentage search time actually in¬ 
creases. Due to the small size of the AOI at the low level, the interdictor is able to get to the 
goal location very quickly releasing the surveyor to continue its search mission in less time than 
in a larger AOI. 

4.3 Sensitivity Analysis 

Having determined that Search Area and Team Type have the largest effect on both response 
variables, we performed sensitivity analysis to quantify those effects. All factors were held 
constant with the exception of Team Type and Search Area. The factor values used are shown 
in Table 4.2 

Both were varied across their respective ranges or categories that resulted in eighteen design 
points. The results shown in Figure 4.12 clearly indicate that having a tracking capability is 
better than no tracking capability. Holding all factors fixed with the exception of Search Area 
and Team Type indicates that as the search area is increased the percentage of targets cleared 
decreases. 

Figure 4.13 also shows that as the Search area increases the search time of the Surveyor also 
increases. 

It is interesting to note that while possessing a tracking capability is superior to no tracking 
capability, there is no statistical difference between the Team Types Surveyor/Tracking and 
Surveyor with Tracker. For both MOEs, it is the tracking effect that affects the MOE, not the 
tracking Team Type. As mentioned in Section 4.2.1, the limiting factor for these Team Types 
is the Interdictor Transit Time. Regardless of how the tracking capability is employed, the 
number of targets cleared in a given mission time is limited by how many targets the interdictor 
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Figure 4.12: Average Percent Targets Cleared vs. Search Area Size 


can actually get to during the mission. 

4.4 Analysis Summary 

Based on the screening for each response, in all cases the factor with the greatest effect is the 
Team Type and Search Area. Surveyor Team Type produces a low response for the Percentage 
of Targets cleared due to the lack of tracking capability. As shown in Figure 4.13, possessing 
a tracking capability reduced the amount of time the Surveyor searches as it performs its sec¬ 
ondary mission of track rather than search. Neither of these conclusions indicate that the QRF 
having an organic Tracker UAV is a valuable asset. The sensitivity analysis performed only 
on the Team Types with a tracking capability produced similar results. For these team types, 
Tracker Launch and Tracker Speed produced no significant effects for either response variable. 
In order to produce an effect in either response variable from Tracker Launch and Speed, we 
would recommend performing the analysis with a Tracker UAV of greater capability. 

We have also discovered that the hard part of target capture is not the interdiction of targets 
but the search for them which relates directly to the Search Area size. How we search and the 
assets available continue be the main limiting factors in target acquisition. The number of team 
types and assets available is an area for future study to help determine an acceptable size AOI 
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Figure 4.13: Average Percent Search Time vs Search Area Size 


for mission accomplishment. For the models above we also found Interdictor Transit Time to 
be a limiting factor for the cases where we studied team types with a tracking capability. The 
QRF can only clear the targets it can get to during the course of a mission and this is limited 
by how fast it can get to each target. The effect of increasing or decreasing Interdictor Transit 
Time is an area worthy of future study. If it makes a difference in terms of the percentage of 
targets cleared, then an analysis of alternatives may be necessary to determine the best form of 
mobility for the QRF of the future. 
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CHAPTER 5: 

Conclusions and Recommendations 


We set out to answer the following research questions: 

• Is it better for an interdiction team to possess an organic “augmented tracking” capability 
in the context of time to intercept and number of targets captured? 

• What are the significant factors that produce teaming strategies that result in the greatest 
number of hostile targets captured? 

The analysis also shows that while the tracking capability is better, the QRF possessing an or¬ 
ganic tracking capability had no effect on either of the MOEs. We also explored three Team 
Types to determine which factors affect the success of those Team Types in terms of the per¬ 
centage of targets cleared and the percentage of time the surveyor searches. Those factors are 
Team Type and Search Area size. 

5.1 Search Area 

It was not surprising to leam that as the Search Area increased, fewer targets were captured. 
This was true regardless of the Team Type employed. While not used as a prediction tool, the 
results of this simulation provide useful insights to a commander who is planning for a mission 
utilizing teams similar to the ones studied here. Based on the given AOI, the commander may 
elect to employ a greater number of teams or partition the AOI into smaller sections and employ 
teams in each partitioned area. 

5.2 Team Type 

Through the analysis, we were able to show that possessing a tracking capability is superior 
to no tracking capability. When tracking was present in the Team Type, the number of targets 
cleared for a given mission is greater than when tracking was absent. Assuming that the Sur¬ 
veyor is national asset and there may only be one available for the AOI and it may be unable to 
track until released by the QRF or Tracker, the team types employed should include a Tracker 
UAV. This would allow the Tracker to be launched to the reported target location and get “eyes 
on” the location to track the target. If the target is not found, the Tracker can then perform a 
search as the QRF approaches in an attempt to reacquire the target. 
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5.3 Future Works 

As previously mentioned, an area for future research is the effects on the MOEs of multiple 
teams in a given AOI. Specifically, a heuristic can be developed that can attempt to maximize 
the number of targets captured by minimizing the time it takes for an interdiction team to arrive 
at a target location. This algorithm can solve a shortest path problem to the target from each 
team, identify the closest team, and then through an interface with SASIO an order will be sent 
to that team to begin the interdiction mission. 

We showed that the Interdictor Transit Time was a limiting factor in the percentage of targets 
captured. An analysis of alternatives can also be conducted to determine the fastest form of 
mobility for a QRF based on current or future systems. A reduction in the transit time should 
lead to an increase in the percentage of targets cleared in a mission. 

Other areas where the SASIO analysis tool would be useful include the Navy’s use of Unmanned 
Underwater Vehicles (UUV) and the USMC study of cargo UAS for battlefield resupply. The 
Navy envisions the UUV to have shallow water capability, stealth, and the ability conduct ISR 
as well as antisubmarine warfare and mine laying operations. The cargo UAS problem will be 
affected by numerous factors including range, payload, altitude and routes. SASIO provides 
the ability to perform a thorough analysis of the factor space for these problems and determine 
which factors will significantly affect the UUV and cargo UAS operations. Once these factors 
are determined a better CONOPS can developed for their use. While these systems greatly 
enhance surveillance capabilities, their greatest contribution will be their ability to aid friendly 
forces in making decisions that will directly lead to an increase in the number of hostile targets 
captured. This is one of the overall goals of these systems. Surveillance is simply a sub-problem 
of the surveillance and interdiction mission set. 
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APPENDIX A: 

NPS-USSOCOM Field Experimentation 


A.l General Description 

Field experiments were conducted to test the various teaming strategies previously described. 
The experimentation was conducted as part of the USSOCOM-NPS Field Experimentation Co¬ 
operative Capabilities Based Experiment (CBE) 10-3 at Camp Roberts Army Reserve Base. 

Assets used included two Raven UAVs to simulate the Surveyor and Tracker UAVs as well as 
three ground vehicles to simulate the blue team QRF and two red team target vehicles. The 
Raven UAV is a Group one UAV according to the Joint-UAS definition. It has a wingspan of 
4ft 3in and a length of 3ft 7in. It weighs 41bs 3oz with a range of 6.2 miles or roughly 10 km. 

Each vehicle was equipped with a radio and handheld global positioning system (GPS) unit to 
record position and the time of significant events. The data was collected at the TOC and stored 
locally with each vehicle for post-experiment playback and analysis. 

The controllable variables for these experiments included Team Type, Search Pattern, and the 
number of objects with the understanding that we would not be able to control the number of 
objects in any AOI. The uncontrollable variables were the remainder of the factors in Table 3.1. 
We did not pick any particular search pattern but instead performed a route search of the Camp 
Roberts road network. This decision was based on the time constraint of the experiment. 

Due to asset and time constraints, the experiment was originally planned to study two teaming 
strategies (Surveyor/Tracking and Surveyor with Tracker) holding all other factors constant. 
An operational thread, or script, was provided to blue and red teams for the conduct of the 
experiment. Multiple experiments were conducted during the allotted time period. After all red 
teams were interdicted or the FOB was attacked, the experiment was reset for the next iteration. 

Operational Lessons Learned 

The results of these field experiments validated the belief that the search effort is hard. Locating 
hostile objects within any size AOI is not an easy task. The results showed that even for a small 
AOI with known routes of travel for red teams, more often than not, the red teams made it to 
the FOB before being interdicted by the blue QRF. Detection of Red Force vehicles by the Blue 
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Figure 1: Raven Launch 


Force patrolling UAV (Raven) was infrequent, limited by field-of-view, unsteady full-motion 
video, and scale of the area of interest (AOI). In some cases the first indication of the presence 
of the red team came from the QRF at the FOB when the red team came within line of sight 
of the FOB. Once the QRF made contact with the red team, either from Raven cueing or self 
cueing, the actual interdiction effort was relatively easy. In today’s very uncertain environments, 
it becomes imperative that the search effort be successful. Future work should focus not only on 
MUT but also on optimizing the search effort to increase the effectiveness of the MUT effort. 

One of the most significant and uncontrollable factors in any tactical operation is the weather. 
These experiments validated the need for the right type of UAV being paired with the appropri¬ 
ate mission. Due to equipment constraints one of the Ravens was used as the Surveyor UAV. 
In some cases, the Surveyor lost track due to head winds reducing its ground speed to the point 
where it was no longer able match the red team velocity and thus the red team drove out of 
the Surveyor field of view and contact was lost. Constraints in real-world settings should be 
considered in decision support models for employment. 

Weather is a valid consideration at the tactical level also. The situation encountered above may 
alter the decision to launch a Raven as a tracking asset if the weather might make it impossible 
for it to track. In these situations, the Surveyor would be required to perform a greater share of 
the tracking mission and thus, we would expect fewer hostile objects to be found. The current 
SASIO model does not take into account weather effects. Weather and its effects on teaming 
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and decisionmaking is an area that should be considered for future study. CBE 10-4 scheduled 
for August 2010 will continue these efforts. 


A.2 Operational Thread Script 

Surveyor / Tracking 

T-0: SURVEYOR launches on a broad area surveillance mission and receives tasking 

from SASIO/Command to search a given location for a mounted target (RED Team 
SUV) 

• Intel developed and passed to SURVEYOR through TOC 

• SURVEYOR: transitions to suspected RED Team location 

• QRF: stationary at FOB awaiting tasking from TOC 

T-1: SURVEYOR locates RED Team SUV 

• SURVEYOR: RED Team location report passed to TOC 

• TOC: updates SASIO:Command 

• QRF: receives tasking from TOCC to interdict RED Team SUV at reported loca¬ 
tion. 

T-2: SURVEYOR transitions to tracking of RED Team SUV 

• SURVEYOR: begins overhead orbit of RED Team SUV. Reports orbit location 
via SASIO: Command 

• TOC: updates SURVEYOR position in SASIO: Command 

• QRF: acknowledges receipt of RED Team location to TOC. QRF begins transition 
to reported RED Team location 

T-3: QRF arrives at reported RED Team SUV location 

• QRF: reports arrival at RED Team suspected location and reports to TOC 

• QRF: identifies RED Team and releases SURVEYOR. Makes report to TOC 

• SURVEYOR: receives new tasking from TOC via SASIO: Command 

• QRF: begins clearing of RED Team. 

T-4: QRF checks for new tasking 

• QRF: reports end of clearing operations to TOC 

• QRF: request additional tasking from TOC 

T-5: If a new report is present, QRF transitions to new RED Team SUV location. If no 

new report, QRF returns to FOB 

T-6: After all RED Team targets are captured or at the end of the experiment, SUR¬ 

VEYOR and QRF will RTB 
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• Collect Data 

o Number of RED targets reported 
o Number of RED targets cleared 
o Amount of time SURVEYOR searched 
o Amount of time SURVEYOR conducted other tasks 

• Collect Video of Mission 

o SURVEYOR video of identified RED target 


Surveyor with Tracker 

T-0: SURVEYOR launches on a broad area surveillance mission and receives tasking 

from SASIO/Command to search a given location for a mounted target (RED Team 
SUV) 

• Intel developed and passed to SURVEYOR through TOC 

• SURVEYOR: transitions to suspected RED Team location 

• QRF: stationary at FOB awaiting tasking from TOC 

T-1: SURVEYOR locates RED Team SUV 

• SURVEYOR: RED Team location report passed to TOC 

• QRF: receives tasking from TOC via SASIO: Command to interdict RED Team 
SUV at reported location 

T-2: SURVEYOR transitions to tracking of RED Team SUV 

• SURVEYOR: begins overhead orbit of RED Team SUV. Reports orbit location 

• QRF: acknowledges receipt of RED Team location to TOCC. QRF begins transi¬ 
tion to reported RED Team location 

T-3: QRF begins transition to reported RED Team SUV location 

• QRF: begins transition to reported RED Team location 

• SURVEYOR: continues orbit over RED Team. Provides video feed to QRF and 
ground station 

• TOCC: updates SASIO:Command with SURVEYOR location and RED Team 
activity 

T-4: QRF launches TRACKER at predetermined distance 

• QRF: launches TRACKER to RED Team location. Reports launch to TOC 

• SURVEYOR: continues overhead orbit and providing RED Team location up¬ 
dates 

• TRACKER: begins transit to RED Team location 
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T-5: TRACKER arrives at RED Team location 

• SURVEYOR: conducts handoff of RED Team with TRACKER 

• TRACKER: assumes tracking responsibilities of RED Team 

• TRACKER: releases SURVEYOR for additional tasking 

• SURVEYOR: receives new tasking from TOC 

• QRF: continues transit to RED Team location 

T-6: QRF arrives at reported RED Team SUV location 

• QRF: reports arrival at RED Team suspected location and reports to TOC 

• QRF: identifies RED Team and releases SURVEYOR. Makes report to TOC 

• TRACKER: RTBs to QRF • QRF: begins clearing of RED Team 

T-7: QRF checks for new tasking 

• QRF: reports end of clearing operations to TOC 

• QRF: request additional tasking from TOC 

T-8: If a new report is present, QRF transitions to new RED Team SUV location. If no 

new report, QRF returns to FOB 

T-9: After all RED Team targets are captured or at the end of the experiment, SUR¬ 

VEYOR and QRF will RTB 

• Collect Data 

o Number of RED targets reported 
o Number of RED targets cleared 
o Amount of time SURVEYOR searched 
o Amount of time SURVEYOR conducted other tasks 

• Collect Video of Mission 

o SURVEYOR video of identified RED target 
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APPENDIX B: 

Primer on Statistical Analysis Using JMP 


Overview 

This appendix will provide the reader with the steps necessary to construct a design matrix in 
JMP [10] and perform screening analysis. This appendix assumes the reader has access to a 
current version of JMP and is familiar with the basic statistical operations. 


Step 1 

Open JMP and select DOE from the menu bar, then select Custom Design 



Figure 2: Choose DOE 


Step 2 

From the custom design dialog window under Factors, select Add Factors 



Figure 3: Add Factors 
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Add all factors and specify their type or role and the associated values. Values can be added 
as coded variables for analysis or the engineering units can be added at the high and low levels 
for input into a simulation model such as SASIO. Once all factors and levels are entered, select 
Continue. 


Step 3 

Select Interactions, then select 2nd to build a design matrix with main effects and 1st order 
interactions. Then select Make Design. 



Figure 4: Interactions selection 


After interactions are selected the model box should show the interactions 
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DOE: Custom Design 

» Custom Design 
Responses 
Factors 

| Add F.a.r. ( Remove ) «<MNraai>rs | II 


Name 

Role 

Changes 

Values 


w Team Type 

Categorical 

Easy 

Surveyor 

Survevor/Tracklnq 

^False Neg Prob 

Continuous 

Easy 

-1 

1 

^False Pos Prob 

Continuous 

Easy 

-1 

: 


Define Factor Constraints 
Model 

( Main Effects Interactions ▼ C RSM ) ^ Cross ^ Powers ▼ ( Remove Term ~) 


Name _ 

Intercept 

Team Type 

False Neg Prob 

False Pos Prob 

Team Type’False Neg Prob 

Team Type*False Pos Prob 

False Neg Prob’False Pos Prob 


Estlmability 

Necessary 

Necessary 

Necessary 

Necessary 

Necessary 

Necessary 

Necessary 


Design Generation 

| 1 Group runs into random blocks of size: 

Number of Runs: 

Q Minimum 

@ Default 8 

O User Specified 8 

' Make Design j 


Figure 5: Model with Interactions 


Step 4 

Before selecting Make Table, select the red triangle next to Custom Design, select Optimality 
Criterion, select appropriate criterion. For this model we chose to use D-Optimality. 



Figure 6: Optimality Criterion selection 


Ensure Run Order is set to Randomize. This is found under Output Options. Now select Make 
Table. 
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O O DOE: Custom Design 

* Custom Design 
Responses 
I Factors 


W ^ Custom 


[E Custom Design 

Design Custom Design 
Criterion D Optimal 

H Screening 
» Model 
- DOE Dialog 

s 

s \ 

Team Type 

False 
Neg Prob 

False Pos Prob 

Y 



Surveyor/Tracking 

1 

1 



2 

Surveyor 

1 

-1 



3 

Surveyor 

1 

1 



4 

Surveyor 

-1 

-1 



5 

Surveyor/Tracking 

1 

-1 



6 

Surveyor/Tracking 

-1 

-1 



7 

Surveyor 

-1 

1 



8 

Surveyor/Tracking 

-1 

1 









E Columns (4/0) 

& Team Type ★ 

A False Neg Prob * 

A False Pos Prob ★ 

A Y* 























































ERows 

All rows 8 

Selected 0 

Excluded 0 

Hidden 0 

Labelled 0 




















































Figure 7: Design Matrix 


Step 5 

After deleting the Y column, the design matrix is now available to exported to Excel or saved 
as a txt file for import into a simulation model. 



Figure 8: Export to Excel 


Select export format. 
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Custom Design 

E Custom Design 

Design Custom Design 
Criterion D Optimal 

^Screening 

- Model 

- DOE Dialog 

’ Select the format to use for exporting this data: 

0 Text 

Creates a text document that can be viewed and edited b> 
- many applications. 

_ © Excel 

_ Creates an Excel workbook that can be opened by Micros 

Excel and other compatible applications. 

Y 



















1 USAS 

Transport LJ Append 

es a SAS Transport data library that can be opened 



E Columns (4/0) 

Crea 



checked, appends this data to an existing SAS Transport 
- data library. 


& Team Type * 

A False Neg Prob * 

A False Pos Prob * 
AY* 










( Cancel Next... 



3 







BRows 







All rows 8 







Selected 0 














































Figure 9: Export Format 
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